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Abstract: This study presents an empirically-based statewide system that links information about 
pre-kindergarten programs with children’s school readiness scores to certify pre-kindergarten 
classrooms as promoting school readiness. Over 8,000 children from 1,255 pre-kindergarten 
classrooms were followed longitudinally for one year. Pre-kindergarten quality indicators of 
intentional instruction, an early literacy focus, and professional development were key predictors 
of kindergarten outcomes. A latent profile analysis identified pre-kindergarten classrooms that 
were high on pre-kindergarten quality indicators and high on kindergarten outcomes (67.3%), 
low on pre-kindergarten quality and kindergarten outcomes (21.3%), or low on quality but high 
on outcomes (11.4%). The last group of classrooms was likely to serve middle-class children and 
not use the state program model. This project demonstrates how a scientific approach can 
inform stakeholders and parents about the effectiveness of early childhood programs. 
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Sistema estatal basado en datos emplricos para identificar programas de educacion pre- 
escolar de calidad 

Resumen: El presente estudio presenta un sistema estatal de bases empfricas que vincula la informacion 
de los programas de pre-kinder con los resultados de la preparacion preescolar para certificar que los 
salones de pre-kinder promueven las habilidades basicas para la escuela primaria. Mas de 8,000 ninos de 
1,255 salones de pre-kinder fueron monitoreados longitudinalmente por un ano. Tres indicadores de la 
calidad en pre-kinder: instruccion intencional, foco en alfabetizacion temprana y desarrollo profesional-, 
fueron claves para predecir los resultados en kinder. Un analisis de perfil latente identified los salones de 
pre-kinder con buenos resultados en los indicadores de calidad y en los resultados de kinder (67.3%), a 
los salones bajos en la calidad de pre-kinder y en los resultados de kinder (21.3%), y a los salones bajos 
en calidad pero altos en resultados (11.4%). Este ultimo grupo de salones eran los mas probables de 
atender a ninos de clase media y de no usar el programa modelo del estado. Este proyecto demuestra 
como un enfoque cientifico puede dar informacion a los padres y a las partes interesadas sobre la 
efectividad de los programas de educacion preescolar. 

Palabras clave: pre-escolar; kindergarten; alfabetizacion emergente; desarrollo social; adecuacion 
para escolaridad; sistema de medicion de calidad; politica de estado. 

Sistema estatal com base em dados empiricos para identificar programas de qualidade pre¬ 
escolar 

Resumo: Este trabalho apresenta um sistema de base empirica estatal que liga informagoes de 
programas de pre-escola com os resultados de preparagao para a escola para certificar que as salas 
de aula do jardim de infancia promovem as competencias basicas para a escola primaria. Mais de 
8.000 criangas de pre-jardim em 1.255 salas de aula foram monitorados longitudinalmente por um 
ano. Tres indicadores de qualidade da instrugao pre-K: instrugao intencional, foco em alfabetizagao, 
e desenvolvimento profissional, foram fundamentals para prever os resultados no jardim de infancia. 
A analise do perfil latente identificados pre-escola salas de aula com bons resultados nos indicadores 
de qualidade e resultados na educagao infantil (67,3%), as salas de resultados de baixa qualidade pre- 
jardim de infancia e creche (21,3 %) e baixa salas de aula de qualidade, mas alta no desempenho 
(11,4%). Este ultimo grupo de salas tinham maior probabilidade de servir as criangas de classe media 
e nao usar o programa do estado do modelo. Este projeto demonstra como uma abordagem 
cientifica pode fornecer informagoes para os pais e as partes interessadas sobre a eflcacia dos 
programas pre-escolares. 

Palavras-chave: infancia, pre-escolar, desenvolvimento da alfabetizagao, o desenvolvimento social, 
aptidao para o ensino, sistema de medigao de qualidade, politica de estado. 

Introduction 

There is increasing recognition of the importance of the early childhood period as a critical 
time for promoting children’s learning in order to assure readiness for success in school (Campbell, 
Ramey, Pungello, Sparling, & Miller-Johnson, 2002; DiPietro, 2000; National Research Council, 
2001). To reach this goal early childhood programs are provided through a variety of service delivery 
models. Public school pre-kindergarten in most states is funded primarily for children from lower 
income backgrounds and federally funded Head Start programs also primarily target very low- 
income families. Childcare programs serve children from all income levels and typically provide 
longer hours of service to meet the needs of working families. Given heightened public awareness of 
the importance of early childhood programs in preparing children to be school ready and the 
multiple service delivery models in which this occurs, systems are needed to determine if, in fact, 
children are school ready after leaving these programs. Recent reports describe how quality rating 
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systems may need to target the performance of programs to advance children’s learning so that 
school ready goals are assured (e.g., foundation skills necessary for reading and social competence; 
Gallagher, Clifford, & Maxwell, 2004; Kagan & Rigby, 2003; Kauerz, 2008). 

States are faced with a number of challenges as they strive to develop early childhood 
education systems that meet the social and academic needs of children, particularly those from low 
income backgrounds, using current funding streams (Kauerz, 2008). As children from low income 
backgrounds are known to have less exposure to rich vocabulary, fewer experiences with books, and 
less parental responsiveness to children’s interests (Evans, 2004; Hart & Risley, 1995; Landry, Smith, 
Swank, Assel & Vellet, 2001), there is increasing belief in the United States that high quality 
preschool programs should be utilized as a means of assuring that children from low socioeconomic 
(SES) backgrounds are prepared to start school in kindergarten. In addition, irrespective of how 
early childhood programs are funded, parents are in need of evidenced-based information regarding 
the extent to which an early childhood program prepares a child to be school ready. 

Shortcomings of Current Quality Rating Systems 

National organizations like the National Association for the Education of Young Children 
(NAEYC, 2008) and the National Association of Early Childcare Professionals (NAECP, 2008), 
conduct evaluations of childcare programs to determine if they meet a broad range of quality 
indicators that, in turn, result in accreditation. Many of the indicators included in these evaluation 
systems are conceptually derived and/or based on long-standing beliefs in the field of early 
childhood, such as the quality of conversations between teachers and children and availability of 
materials for children’s exploration. Accreditation from these national organizations can provide 
important information to families about quality indicators of a program. However, many preschool 
programs are not able to apply for accreditation because of the costs involved. For example in many 
states, only a small percentage of childcare and Head Start programs seek these accreditations. 
Additionally, few public school programs seek accreditation by national organizations. In light of 
this, state education leaders and policy makers are looking for cost-effective approaches to 
document whether early childhood programs from various service delivery models are adequately 
preparing children for kindergarten. 

Several states across the nation (Colorado, Washington, Pennsylvania, Oklahoma, North 
Carolina, and Ohio; Zellman & Perlman, 2008) have implemented quality rating systems for early 
childhood education, ( e.g.. Zellman. Perlman. Le. & Setodii. 20081 However, although recent reports 
emphasize the need for outcome-based accountability and information about program effectiveness 
(Kaurtz, 2008; Zellman, Brandon, Boiler, & Kreader, 2011), only two systems (Colorado’s Qualistar 
Rating System and Washington’s Seeds to Success) directly link accreditation to the extent to which 
a preschool program has actually provided children with the skills necessary for success in 
kindergarten. To do that, an evaluation system must assess characteristics of program quality and 
relate those characteristics to children’s learning. Current accreditation procedures that do not 
consider this link may involve a great deal of time and financial resources in the process of capturing 
program, classroom, and teacher characteristics that are not meaningfully related to children’s school 
readiness. In the initial development phase of the Texas School Ready system described here, policy 
makers prohibited the use of child-level outcome data during the preschool year due to concerns 
about high-stakes testing of preschool children. However, schools in Texas are required to 
administer an approved literacy screener within the first six weeks of children’s entry into 
kindergarten. No current evaluation system considers the link between preschool quality and 
children’s school readiness when they are actually entering school. 
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Rationale for Development of a New Quality Rating System 

There are many reasons for identifying which early childhood programs are effective in 
preparing young children for school. While recent large scale reports indicate that high quality early 
childhood centers can be as cognitively stimulating as home care in middle-class families, working 
parents of lower income have less access to such high quality centers (Phillips, Voran, Kisker, 

Howes, & Whitebook, 1994). Moreover, it is high quality, early childhood programs that have 
demonstrated a positive influence on children’s school readiness (e.g., Bierman, Domitrovich, Nix, 
Gest, Welsch, Greenberg, et al., 2008; Hindson, Byne, Fielding-Barnsley, Newman, Hine, & 
Shankweiler, 2005; Reynolds, Ou, & Topitzes, 2004). Thus, one of the most important reasons for 
identifying effective programs is to provide parents with information that they can use to guide their 
decisions regarding selection of the best program for their child. Of concern is that families at 
poverty levels are more likely to rely on center-based programs that accept federal subsidy (Center 
for the Childcare Workforce, 2004) and such programs tend to have very limited funding earmarked 
for quality enhancement. Thus, programs that accept federal subsidy tend to suffer from underpaid 
and poorly trained staff and a dearth of appropriate learning materials (e.g., books, puzzles, blocks). 
As a result, young children from low SES homes, who are at greatest risk for developmental 
difficulties and scholastic unpreparedness, are spending time in early childhood centers that may be 
less likely to provide high quality stimulation and responsiveness as part of their instructional 
practices. 

A second compelling reason to develop quality rating systems for early childhood programs 
that are linked to child outcomes is for accountability and program development. Legislators and 
others responsible for funding are beginning to insist on evaluative information in order to sustain 
or increase program funding. Additionally, those responsible for program management and 
implementation (e.g., director, principal, teacher) could use evaluative information to inform their 
educational practices. While self-assessment is often conducted, the extent to which this is used to 
enhance program quality is not well established. This often occurs because it is not collected in a 
form that can be used to inform and guide programmatic changes. An optimal quality rating system 
would provide programs that are in need of improvement with an individualized training and 
technical assistance plan. 

While there are numerous reasons for using quality rating systems, there are also many 
challenges. For example, funding for rating systems on a statewide basis is often restricted, and 
therefore, approaches need to be efficient. Efficiency means finding methods that are reliable and 
valid but at the same time cost-effective. For example, whereas direct observations of preschool 
programs and classrooms over multiple visits are currently the norm, these methods are costly with 
respect to travel and personnel time. If programs must bear these costs then the expense is often not 
feasible for small programs and those whose operations are dependent upon federal subsidy. Also, 
rating systems that evaluate program quality need to be equally applicable across all types of service 
delivery systems (e.g., public school, Head Start, childcare). For example, a good quality rating 
system must not be biased for or against programmatic differences that exist institutionally as a 
function of service delivery systems. Otherwise, the quality rating system would not be fully 
endorsed or implemented, which would negate the potential benefit of being able to compare 
preschool programs that function under different auspices. 

Sociopolitical Context of the Present Study 

The goal of the present study is to describe the development of an empirically-based quality 
rating system for early childhood programs, an effort directed by a state legislated request to first 
integrate delivery of early childhood education services for preschool-age children from 
economically disadvantaged families. This 2003 request first encouraged early childhood education 
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programs (i.e., child care, school districts. Head Start) to come together in a seamless mixed delivery 
model and adopt similar standards and practices, and a second legislative request (2005) was to 
develop quality rating systems. Approximately 1000 pre-kindergarten classrooms in Texas voluntarily 
participated in the integrated service delivery model during the time of the present study. The three 
types of early childhood programs have differences in policies about service focus in areas such as 
education, health, and social services (Gallagher et al., 2004). However, the emphasis in this new 
integrated service delivery model, called the Texas Early Education Model (TEEM; Landry, Swank 
Anthony & Assel, 2011), is on preparing children to be school ready upon entrance into 
kindergarten by providing them with the foundational language, literacy, mathematics, and social 
skills necessary to succeed in kindergarten. Programs enrolled in TEEM share a common set of 
supports, including professional development, provision of curricular materials, training in a 
research-based curriculum, a system for assessing children’s progress in cognitive and social skills, a 
set of state learning standards, and classroom mentors. A recent paper described this integration 
model and its efficacy in terms of both changes in preschool teachers’ instructional practices and 
children’s school readiness (Landry et al, 2011). 

Purpose of the Present Study 

The primary objective of this report is to describe the development of a scientifically-based 
and empirically-derived system that is being used across Texas to rate the quality of pre-kindergarten 
classrooms. The School Readiness Certification System (SRCS) is not intended to replace other 
accreditation standards, such as those offered by NAEYC or state licensing bodies. Instead, it was 
designed to provide specific information about how programs prepare pre-kindergarten children for 
school so that parents can make informed decisions about early childhood education programs. The 
SRCS also was designed to provide information to guide technical assistance for those programs that 
do not meet certification criteria. In addition, the SRCS was designed to have the advantage of being 
equally available to all types of early childhood program providers, adding to its utility and provision 
of greater choice to parents. Finally, the SRCS was developed under the guiding notions that the 
rating system should be objective and evidence-driven and that it should maintain a concentrated 
focus on indicators that predict school readiness. 

The theoretical, conceptual, and practical underpinnings of the SRCS were developed in 
conjunction with state educational leadership, national experts in early childhood, and community- 
based stakeholders (e.g., directors, teachers, parents). The agreed upon objective was that the SRCS 
should be a statewide system, applicable across service delivery systems, that incorporates 
information about the quality of pre-kindergarten programs in combination with children’s early 
reading and social competencies, which are assessed by the public schools when children are in 
kindergarten. For the SRCS to be feasible for implementation across a large state with funding 
constraints, it had to be cost-effective yet still reliable and valid. These aims and constraints required 
that the development of the SRCS involve multiple steps, including (a) development of a web-based 
application system for pre-kindergarten programs, (b) documentation of the reliability and validity of 
data provided by early childhood program staff, (c) determining the feasibility of various procedures 
for tracking children from pre-kindergarten into kindergarten, (d) determination of which pre¬ 
kindergarten variables reliably predict kindergarten school readiness in this population, and (e) use of 
statistical models to ascertain the best criteria for certifying classrooms as “School Ready”. The 
present report briefly summarizes the development stage and two pilot studies for the sake of 
completeness, while it fully details the scientific underpinnings used to establish the certification 
criteria. 
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Methods 


Pilot Studies 

Development and validation of the Teacher Self-Report survey. The first pilot study was conducted 
during the 2004/2005 academic school year for the purpose of developing and validating a set of 
survey questions that teachers could answer as part of a web-based application for certification. A 
pool of 74 items was generated based on current research and best practices in the pedagogy of 
promoting early childhood learning across social, language, literacy, and math domains. Included 
were questions about responsive teaching practices, classroom arrangement and organization, daily 
routines, lesson planning, monitoring progress of children’s learning, use of small vs. large group 
activities, and classroom curriculum and materials. The broad item content required individual items 
to vary in scaling metric, and no scale structure was presumed because of the newness of the survey. 
Ninety-nine teachers completed the survey online. 

Given that certification would be based, in part, on teachers’ self-report of their teaching 
practices and characteristics of their classrooms, it was imperative to validate data gathered from the 
Teacher Self-Report survey with independent observations. Trained research assistants observed the 
99 teachers who completed the Teacher Self-Report. Following the standardized procedures 
prescribed for the Teacher Behavior Rating Scale (TBRS; Landry, Crawford, Gunnewig & Swank, 2002), 
observations lasted approximately two and one half hours during the time of the day that was 
devoted to instruction aimed at cognitive and school readiness skill acquisition. The TBRS is a 101 
item, standardized classroom observation system that has been used in a number of state and 
national program evaluation projects (e.g., national evaluation of Early Reading First). Observers 
rate the quality and frequency of occurrence of specific teaching behaviors known to predict 
children’s school readiness. The teacher behaviors that are rated fall into twelve conceptual 
categories, namely oral language, print & letter knowledge, phonological awareness, written 
expression, book reading, centers, mathematics, lesson planning, team teaching, progress 
monitoring, portfolios, and classroom management and responsive teaching practices that are not 
content specific. Observers rate the frequency of occurrence of each target behavior (i.e., 0-1 
occurrence, 2-3 occurrences, or 4 or more occurrences). Observers also rate the quality of each 
target behavior as low, medium, or high. Thus, there are separate quality and frequency summary 
scores for each of the twelve scales of the TBRS. Inter-rater reliability using generalizability 
coefficients (Mitchell, 1979) range from .80 to .98, and internal consistency, for the measure as a 
whole, using Cronbach’s alpha is = .96 (Landry et al., 2002). 

Zero-order correlations of TBRS scales with questions from the Teacher Self-Report were 
tested for significance. In general, questions from the Teacher Self-Report were more highly 
correlated with conceptually similar TBRS scales than with conceptually dissimilar TBRS scales. 
Those survey questions that had three or more significant correlations above r = .25 with 
conceptually overlapping TBRS scales were retained and left unchanged. Survey questions that 
yielded significant but small correlations with conceptually overlapping TBRS scales were reviewed 
for possible sources of confusion and if the source was obvious then they were reworded and 
retained. Survey items not reliably correlated with any like TBRS scale were dropped. Most 
constructs measured by the Teacher Self-Report were indexed by a number of survey items that did 
demonstrate good convergent validity with similar TBRS scales. There were two exceptions. None 
of the Teacher Self-Report items that inquired about responsiveness to children’s needs (learning, 
emotional, or otherwise) or behavioral management of the classroom demonstrated convergent 
validity with like TBRS scales. Thus, in general, results provided preliminary validation that teachers 
were accurately reporting their instructional and classroom practices in ways that could be 
corroborated by an independent observer. 
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Feasibility ofprocedures for tracking children and obtaining assessment data. Feasibility studies were 
conducted during the 2005/2006 school year. Given the many possible challenges to linking pre¬ 
kindergarten school data with children’s kindergarten outcome data, it was necessary to examine the 
practical and fiscal feasibility of various procedures. For example, a variety of methods of tracking 
children through their pre-kindergarten and kindergarten years were tested. This often involved 
children transferring among multiple schools. To accomplish tracking of children, coordination of 
efforts were required among the Texas Education Agency, regional education service centers, 
independent school districts, early childhood education agencies, and Head Start grantees. We also 
piloted a variety of methods aimed at collecting kindergarten assessment data from kindergarten 
teachers, elementary schools, independent school districts, and third party vendors. Examples of 
different methods included sharing of electronic data, development and sharing of protected Excel 
templates, testing of individual children by research staff, and a third party secured web-based 
application. Ultimately, kindergarten assessment data were gathered from 774 children who attended 
20 public school districts across the state. 

The feasibility studies revealed that state legislation was needed to increase schools’ 
compliance with Texas Education Agency’s recommendation to provide kindergarten assessment 
data. It was also determined that a web-based application was optimal for gathering kindergarten 
assessment data. For example, restrictions could be programmed in a web-based system to require 
uniform reporting of assessment data, which was determined necessary given that schools around 
the state were recording the assessment data in different ways (e.g., item level scores, subtest raw 
scores, categories of competency across the entire screening measure). Finally, the feasibility studies 
demonstrated that it would be more cost effective to have the state education agency perform a large 
scale matching of pre-kindergarten children’s demographic data with those of enrolled 
kindergarteners the next year than to have regional education service centers explore matches 
individually. 

Participants in the Main Demonstration Project 

Participants included 538 preschool sites, 1,326 pre-kindergarten classrooms, and 12,585 
preschool-age children. Preschools were distributed across three types of service delivery systems; 
258 childcare, 147 public school and 133 Head Start schools. In each community partnership there 
was representation of classrooms from each type of delivery system. The lead agency in the 
community partnership (i.e., public school, Head Start) held regular meetings to inform the 
participating programs about the application process. The Head Start programs represented all types 
of federally funded grantees, including Community Action, school district, and State Region Service 
Center programs. The childcare programs included for-profit and not-for-profit and ranged from 
small centers to large YMCA programs. Of the 1,255 classrooms from which data were gathered, 
1,024 were participating in the voluntary, state integration program model that focused on programs 
serving low-income children. The 231 classrooms that did not participate in this model were invited 
to participate in the current study to increase the heterogeneity of the sample, as these 231 pre¬ 
kindergarten classrooms served children from middle-income families. 

In the spring of 2006, the preschool programs submitted an on-line application that included 
information detailed in the Procedures section. Programs were aware that though this was a 
demonstration year, their participation would result in a decision about their certification status. In 
the fall of 2006, school readiness scores (reading and social screening measures) were obtained from 
kindergarten classrooms for approximately 8,000 (73%’) of the children from these preschool 


1 Many children attended kindergarten in districts where an assessment other than the Texas Primary Reading 
Inventory (TPRI) or Tejas LEE (TJL) was administered so these children’s data could not be used, and some 
children’s data did not contain enough information for a definitive match. 
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classrooms. Fifty-one percent of the children followed were boys. Children’s average age on the first 
day of kindergarten was 5.5 years (SD = .30). The number of days children attended preschool 
ranged from 1 to 250, with an average of 156 days (SD = 35.5). The sample of children was 
ethnically diverse; 69% Hispanic, 17% Caucasian, 13% African American, and 1% American Indian, 
Asian, or Pacific Islander. A large percentage of children (i.e., 67%) were eligible for free and 
reduced lunch and 48% had limited proficiency in English. 

Procedures 


e-School Plus 

The web-based application had three components: (a) preschool facility report, (b) manage 
my school and students, and (c) teacher self-report. Information gathered in the three components 
included characteristics of the preschool program and preschool classroom, teaching practices, and 
teaching beliefs, based on empirical research that has identified components associated with high 
quality programs (National Early Literacy Panel, 2008). For example, information was gathered 
concerning aspects of the classroom environment (e.g., availability of small group learning areas, 
space for children to gather for a large group activities, literacy richness)(Crosser, 1992; Dunn, 
Beach, & Kontos, 1994), instructional practices (e.g., intentional language and literacy instruction as 
well as time for children to explore and practice for mastery; National Research Council, 2001; 

Raver & Knitzer, 2002), curriculum (e.g., availability of research based scope and sequence for 
cognitive and social learning; Assel, Landry, Swank, & Gunnewig, 2007; Whitehurst, Epstein, Angell, 
Payne, Crone, & Fischel, 1994), assessment approaches (reliable and valid information about 
children’s learning across the year with feedback to teachers to direct instruction; Phaneuf & 
Silberglitt, 2003), and professional development (e.g., small group, learning across time, support for 
opportunities to practice in the classroom; Elmore, 2002; National Commission on Teaching & 
America’s Future, 1996). The application included the following components. 

Preschoolfacility report. The director of a child care or Head Start program or principal of a 
public school pre-kindergarten program provided information on key areas of their program 
including: (a) accreditation/ licensing, staffing patterns (i.e., teacher education, number of teachers 
and teaching assistants), (b) staff credentials (i.e., state certified, bilingual, early childhood, English as 
a second language, reading specialist, child development associate-CDA), (c) program/classroom 
characteristics (i.e., full vs. half day, language of instruction, number of books), (d) curriculum used 
for literacy, math and social/emotional development, (e) methods and frequency of assessing 
children’s academic knowledge (e.g., standardized, work sampling, checklist, Personal Digital 
Assistance), (f) professional development practices, and (g) community integration. This component 
required about 15 minutes to complete. 

Manage my school and students. Each preschool program was asked to provide information on 
their classroom teachers and those children who would be going to kindergarten the following year 
(e.g., number of preschool classrooms, length of program day, and language of instruction). 
Demographic information was entered for each child (e.g., age, gender, home language) as well as 
information related to the educational experience (e.g., eligibility for free/reduced lunch, special 
education, English language learners). 

Teacher Self-Report survey. Each pre-kindergarten teacher was asked to provide information on 
highest level of education and credentials as well as respond to a series of questions regarding 
practices related to the following: (a) best practices, (b) classroom arrangement, (c) lesson planning, 
(d) child monitoring and assessments, (e) social skills development, (f) book reading, (g) 
phonological awareness, (h) print and letter knowledge, (i) written expression, (j) math, and (k) oral 
language instruction. For example, the teacher was asked to respond to a group of questions 
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regarding techniques for monitoring children’s progress and teaching practices for book reading, 
phonological awareness, letter knowledge, and written expression. Other questions required the 
teacher to choose from an array of pictures those that were similar to activities they used for 
promoting early literacy skills, early mathematics skills, and classroom organization. In addition, 
teachers’ beliefs regarding children’s needs for standardized classroom practices versus 
individualized instruction were included. Such questions inquired about the extent to which 
individualization of lesson plans and behavior management techniques were used. Completion of 
the Teacher Self-Report required about 20 minutes. 

Teachers also submitted examples of their daily schedules and lesson plans via US Postal 
Services. The daily schedules and lesson plans that teachers submitted varied tremendously in 
amount of detail and intelligibility. As could be expected, teachers approached lesson planning and 
labeling of daily schedules very differently. For example, while some teachers specified learning 
objectives and correlated activities in their lesson plans, other teachers’ lesson plans resembled daily 
schedules that simply listed a time of day and vague classroom activities (e.g., circle time). In short, 
our coding team concluded these data were not amenable to coding for potential inclusion in the 
certification criteria. 

Each teacher submitted 12 photographs of their classroom, either via uploading of digital 
photographs or via mailing of disposable cameras. Specifically, teachers submitted photographs of 4 
walls, 4 corners, and 4 areas of their choice. A 15-item coding system was developed based on the 
Preschool Classroom Environment Checklist (CIRCLE, 2001). Using two- or three-point rating 
scales, research assistants determined the presence, amount, and quality of the classroom 
arrangement, centers, themes, children’s books, writing materials, labels, children’s names and work 
samples, samples of modeled writing, letter wall, daily schedule, helper chart, and props for group 
instruction (a description and scoring examples can be found in Appendix A). Research assistants 
received two days of training, and were certified for coding once an average agreement with an 
expert coder exceeded 92% for each item. Raters then attended bi-weekly coding meetings to 
discuss scoring questions and prevent coder drift. Ten percent of the pictures were randomly 
selected for recoding by a different rater to ensure inter-rater reliability throughout the coding period 
with reliability exceeding 0.80 (Mitchell, 1979). 

Kindergarten School Readiness Outcomes 

To identify each child’s kindergarten teacher, the first step was to fund the 22 state regional 
education service centers to perform a series of searches in which children’s information from e- 
School Plus was matched to children’s information included in the Texas Education Agency’s 
centralized databases. Information used to match children’s data included social security number, 
when available, and children’s demographic data, including name, gender, ethnicity, birthday, 
free/reduced lunch status, etc. When this matching process yielded a partial match, decisions were 
made on a case-by-case basis. When a student’s information matched two records in the TEA’s 
database equally well (i.e., partial match with both records), neither was used. Matches yielded the 
name of a school that a given child was believed to attend. Next, a unique identifier was assigned to 
each “matched” child who attended public or charter schools in the state. 

Schools that included children “matched” through this process were sent a letter and asked 
to provide the names and contact information of the child’s kindergarten teacher. These teachers 
were contacted via the US Postal Service, email, and telephone and provided with a password and 
instructions for how to login to e-School Plus and how to enter children’s school readiness data. 
School readiness data included scores from the district administered reading screenings (see below) 
and kindergarten teachers’ responses to a social competence screener (see below). Kindergarten 
teachers’ access to e-School Plus was restricted in such a way that they only saw a list of names of 
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children who attended their particular kindergarten classroom. Finally, because preschool classroom 
was the unit of analysis for school readiness certification (see below), the kindergarten outcome 
scores for all children within a given preschool classroom were averaged to create classroom-level 
pass-rates (Snijders & Bosker, 1999). 

Texas Primary Heading Inventory. About 90% of school districts and charter schools in the state 
of Texas administer the Texas Primary Reading Inventory (TPRI; Texas Educational Agency, 1998) 
two to three times per school year during kindergarten, first grade, and second grade. This measure 
has strong reliability and validity and has been demonstrated to predict end of second grade reading 
(Texas Educational Agency, 2002). For each progress monitoring wave, the inventory includes a 
screener and a more comprehensive inventory. The kindergarten screener consists of two subtests 
that permit rapid determination of whether or not students are at-risk for reading problems at the 
end of first grade. Specifically, the Graphophonemic Knowledge subtest requires children to provide 
the sounds associated with each of ten letters. The Phonemic Awareness subtest requires children to 
(a) blend onset and rime units into real words and to (b) blend up to three individual phonemes into 
real words. Standard administration procedures dictate that if a child scores “Developed” on the 
Graphophonemic Knowledge subtest, then the child is classified as “Developed” on the screener 
and the Phonemic Awareness subtest is not administered. Flowever if a child does not score 
“Developed” on the Graphophonemic Knowledge subtest, then he or she is administered the 
Phonemic Awareness subtest, and if the child achieves a score of “Developed” on the Phonemic 
Awareness subtest then he or she is classified as “Developed” on the screener as a whole. In the 
present study, children’s classification status on the screener was used as the reading outcome 
because only the screener is administered to all children. Because some schools in the state did not 
administer the TPRI as part of their standard operating procedures, research staff individually 
administered the TPRI to approximately 600 children. In total, 76% of the sample followed took the 
TPRI, and the average preschool classroom pass-rate was 65%. 

Tejas Tee. Most children in Texas who speak Spanish in their homes and who are provided 
classroom instruction in Spanish are administered the Tejas Lee (TJL; Texas Educational Agency, 
2002) instead of the TPRI. The TJL, like the TPRI, is used to monitor children’s learning of literacy 
skills two to three times per school year during kindergarten, first grade, second grade and third 
grade. The TJL is not a Spanish translation of the TPRI, however it does assess similar literacy 
component skills. Additionally, the TJL was developed following procedures that closely resembled 
those used to develop the TPRI. Because the 2006/2007 TJL did not include a screener, a scoring 
routine that conceptually paralleled that of the TPRI was used to classify children. Specifically, 
priority was given to the letter sounds subtest such that if children scored “Developed” on this 
subtest then children were classified as “Developed”. If not, then children were still classified as 
“Developed” if they achieved a score of “Developed” on the subtests that involved blending of 
syllables and segmentation of syllables. In total, 24% of the sample followed took the TJL, and the 
average preschool classroom pass-rate was 82%. 

From the TPRI and TJL scores, a single classroom-level Literacy pass-rate was constructed 
by first taking each child’s score, regardless of measure, and then calculating the classroom average. 
Although there are difficulties with aggregating scores from different measures, this was necessary to 
avoid running two sets of certification analyses, which could have potentially created different 
certification criteria for the classrooms serving English- and Spanish-speaking populations. In 
addition, because children from the same preschool classroom could be assessed with different 
reading screeners in Kindergarten, conducting separate analyses could have resulted in different 
certification decisions for the same classroom. 

Social competence and behavior screener. The social competence subscale from the Social 
Competence and Behavior Evaluation (SCBE-30; Dumas, Martinez, & LeFreniere, 1998; Kotler & 
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McMahon, 2002; LaFreniere & Dumas, 1996) was used to evaluate children’s social skills with peers 
and teachers in early kindergarten. Specifically, this 10-item, teacher-report checklist measures 
children’s social behavior, such as social integration, tolerance, and cooperation (e.g., comforts or 
assists other children in difficulty, works easily in a group). Each item utilizes a six-point scale 
(l=almost never occurs to 6=almost always occurs). The SCBE-30 has demonstrated high inter¬ 
rater reliability (.83-.87) and good test-retest reliability (.82). The SCBE-30 has been successfully 
validated and used in numerous studies for monolingual as well as for children learning English as a 
second language (e.g., LaFreniere & Dumas, 1996; LaFreniere, Masataka, Butovskaya, Chen, Dessen, 
Atwanger, et al., 2002). 


Results 

Analysis proceeded in three stages. Stage One involved reducing the number of variables 
from the various components of e-School Plus. This was necessary because the facility report, 
teacher self-report, and classroom picture data comprised over 100 variables. Exploratory and 
confirmatory factor analyses were performed to yield a more manageable number of variables. In 
Stage Two, the resultant factors were combined with the kindergarten outcome data (i.e., classroom- 
level reading and social screener averages) and a latent profile analysis was conducted to identify 
classroom characteristics that were associated with children’s school readiness as assessed in 
kindergarten. Because not all classrooms in a facility entered data into the system, classroom was 
deemed to be the most appropriate unit of analysis in order to prevent certifying a school based on 
partial information. Finally in Stage Three, follow-up analyses were conducted to examine the 
composition of the latent profiles in terms of service delivery systems (i.e., Head Start, public school, 
child care) and participation in the state’s integration model (i.e., TEEM). 

Data Reduction 

Overall strategy. Items from e-School Plus and items from the coding system that described 
the photograph data were subjected to separate exploratory factor analyses. Prior to factor analysis, 
all variables were examined for evidence of non-normality, and in cases in which only mild to 
moderate skewness was observed, model parameters were estimated using maximum likelihood with 
robust standard errors. In some instances, values of one or more of the variables in the analysis had 
to be collapsed due to low response frequency. These variables were modeled as ordinal, rather than 
interval, and parameters were estimated using mean- and variance-adjusted weighted least squares 
estimation (Muthen, du Toit, & Spisic, in press). Varimax or Promax rotation was used, based on the 
size of factor correlations. Once the best exploratory model had been determined, confirmatory 
factor analysis was used to generate factor scores for each factor retained in the EFAs. This 
procedure was used to optimize interpretability of factor scores, by permitting no crossloadings. 

Teacher Self-Report survey. Eight items from the teacher survey were dropped because they did 
not correlate with any other variables (four items about book & print awareness, one about 
classroom arrangement, one about lesson planning, one about print and letter knowledge, and one 
about the number of teacher aides) and an additional four items were dropped because they failed to 
produce unique loadings that exceeded .30 on any factor. Exploratory factor analysis 2 of the 
remaining 37 items yielded eight correlated factors with eigenvalues greater than 1.0. The scree plot 
was consistent with either a three factor solution or a six factor solution. The six factor solution was 


2 For these and all other analyses of teacher and classroom variables, multilevel analyses (teacher/classroom 
nested within school) were attempted. However, due to the preponderance of schools for which only one 
teacher or classroom existed in the data (49%), these models consistently failed to converge or had problems 
with the parameter estimation. Therefore, traditional, single-level analyses were conducted. 
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judged interpretable, and yielded a root mean square error of approximation (RMSEA) of .04, which 
reflects a good fit when considering the complexity of the model (see Table 1 for factor reliabilities 
and Table 2 for description of the six survey factors). 


Table 1 

Factor Reliabilities 


Section 

Factor 

Number of 
items 

Cronbach's 

alpha 

Teacher Survey 

Shared reading 

4 

0.53 

Teacher Survey 

Math instruction 

7 

0.87 

Teacher Survey 

Assessment 

3 

0.49 

Teacher Survey 

Intentional instruction 

9 

0.57 

Teacher Survey 

Early literacy 

8 

0.76 

Teacher Survey 

Lesson planning 

6 

0.41 

Classroom pictures 

Center quality 

6 

0.59 

Classroom pictures 

Visual supports 

4 

0.50 

Professional development 

Core training 

4 

0.76 

Professional development 

Health and safety 

5 

0.85 

Professional development 

Teaching techniques 

3 

0.64 

Assessment 

Checklists 

3 

0.88 

Assessment 

Organization-developed 

3 

0.93 

Assessment 

Curriculum-based 

3 

0.91 

Assessment 

Handheld / PDA 

3 

0.74 

Assessment 

Standardized tests 

3 

0.87 

Assessment 

Work sampling 

3 

0.84 


Classroom pictures. Three items from the photograph-based Preschool Environment Checklist 
were dropped because of low inter-rater reliabilities (classroom labels, presence of educational 
posters/wall materials, and organized classroom theme). Two additional items were dropped 
because they failed to produce unique loadings in excess of .30 on any factor (children’s work 
displayed and samples of modeled wridng). Kappa statistics for the remaining 10 items ranged from 
.56 to .94. Exploratory factor analysis yielded three correlated factors with eigenvalues greater than 
1.0. However, the scree plot was consistent with a 2-factor solution, the 3-factor solution included a 
one-item factor, and the two-factor solution yielded a reasonable fit (RMSEA=.06). Therefore, two 
factors were retained. 

The six items that comprised Factor 1 inquired about the presence of labels, children’s 
names, and other visual supports used for group instruction or classroom management (e.g., 
calendar, weather chart, daily news, letter wall, helper chart, daily schedule). Therefore, Factor 1 was 
labeled Visual Supports. The four items that formed Factor 2 asked about the quality of centers and 
how well supplied these were with literacy materials (e.g., books, writing supplies). Factor 2 was 
therefore labeled Center Quality. The two factors were correlated at .45. 

Facility-report - Professional development. The Facility Report of e-School Plus included fourteen 
items that inquired about the kinds of professional development provided at a given facility. One of 
these items was dropped because it failed to produce unique loadings in excess of .30 on any factor. 
Exploratory factor analysis of the 13 remaining items yielded three correlated factors with 
eigenvalues over 1.0. This three-factor solution was the most interpretable among alternative 
models, and it fit the data well (RMSEA = .06). 
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The first factor included five items that inquired about professional development that was 
focused on teaching of core subjects (e.g., language, math). Factor 1, therefore, was labeled 
Professional Development in Core Subjects. The second factor was comprised of three items that 
inquired about professional development that was focused on teaching methods/techniques (e.g., 
classroom management) and was labeled Teaching Techniques. Finally, the five remaining items 
(e.g., CPR training, nutrition, health & safety) formed the third factor, labeled Flealth & Safety 
Training. Inter-factor correlations ranged from .49 to .59. 

Facility-report - Assessment. The Facility Report of e-School Plus also included 18 items that 
inquired about the frequency that organizations administered different types of assessments to assess 
children’s language, literacy, mathematics, and social/emotional/behavioral competence. 

Exploratory factor analysis of all 18 items yielded six factors with eigenvalues greater than 1.0. Each 
factor was comprised of three items. The six factor structure mapped perfectly onto the six 
assessment methods. Thus, there were separate factors for Standardized Tests, Progress Monitoring 
with PDA, Work Sampling, Checklists, Curriculum-Based Measures, and Organization Developed 
Measures. Substantively, these results suggest that organizations tended to use the same assessment 
method across domains of school readiness and across the age range of preschool children. 
Correlations among the six assessment factors ranged from .04 to .58 (median = .34). Although the 
six-factor solution was readily interpretable, residual variances were substantial, as indicated by a 
relatively high RMSEA (.16). This is likely due to the very high correlations among assessment 
methods increasing the power of the analysis. However, the Root Mean Square Residual was 
reasonable (.04). 

Correlation of factor scores with kindergarten outcomes. Because kindergarten school readiness was 
considered a primary outcome of interest that should inform the SRCS, the final step in reducing the 
number of variables for the latent profile analysis was to check the zero-order correlations of the 
school readiness outcomes with all the factor scores generated above and any remaining variables 
from e-School Plus that were not part of the data reduction thus far (e.g., average number of books 
in the classroom). Any preschool variable that was not correlated with at least one of the 
kindergarten school readiness outcomes was excluded from subsequent analyses. School readiness 
outcomes were based on children’s social screener scores and their classification status obtained on 
the literacy screeners. Children’s school readiness outcomes were aggregated to the level of 
preschool classroom so that they could be correlated with classroom level indicators of preschool 
quality. Thus, kindergarten variables were the percent of children in a given pre-kindergarten 
classroom who scored “Developed” on literacy screeners and the average social screener score 
among children from a given pre-kindergarten classroom. Three indicators of preschool quality were 
significantly correlated with at least one indicator of school readiness: Intentional Instruction from 
the Teacher Self-Report, Early Literacy Instruction from the Teacher Self-Report, and Professional 
Development in Core Subjects from the Facility Report. 

Identification of Latent Profiles of Classrooms 

Our ultimate goal was to determine criteria for certifying or not certifying classrooms that 
had entered the SRCS. In other words, based on the results of the analyses, each classroom would 
have to be assigned to either a certified group or a not-certified group. Therefore, we decided that a 
clustering approach, in which classrooms were grouped into different categories based on the 
pattern of their responses, was most appropriate. Specifically, latent profile analysis (LPA) was used. 
Similar to factor analysis, LPA is a latent variable approach in which the relations among observed 
variables are explained in terms of an unobserved (latent) variable. Unlike factor analysis, the latent 
variable in LPA is discrete, rather than continuous, and the focus of the analysis is the classification 
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of cases (in this case, classrooms) into latent profiles based on their pattern of means on the 
dependent (endogenous) variables (Magidson & Vermunt, 2004). 

Table 2 


Summary of ¥ actors from the Teacher Self-Report Survey 


Factor 

Type of items 

Number of Items 

Factor label 

1 

Strategies used to promote oral language 
development during shared book reading. 

4 

Shared Reading 

2 

Materials used to teach mathematics concepts. 

7 

Math Concepts 

3 

Frequency and kinds of child assessments 
administered and how they are used. 

3 

Assessment 

4 

Intentional teaching strategies that crossed 
content areas. 

9 

Intentional 

Instruction 

5 

Frequency that teachers employed various 
strategies that promote early literacy (e.g. 
writing and print awareness. 

8 

Early Literacy 
Instruction 

6 

Relative importance of teacher directed 
instruction/lesson planning versus child 
initiated play/following children’s interests; 
frequency that teachers used their own ideas, 
district standards, agency guidelines, and state 
standards to guide their lesson planning. 

6 

Lesson Planning 


The three classroom-level factor scores and the two kindergarten outcome variables, which 
were aggregated to classrooms, were entered into the latent profile analysis. Of the original 1,225 
classrooms in the system, 1,185 classrooms had data on at least one of these factors or kindergarten 
outcome variables, and 447 were missing at least one data point. The most common pattern of 
missing data (69%) was missing both the facility report and kindergarten outcome data. As long as 
the data are missing at random (MAR), maximum likelihood estimation techniques using full 
information will produce unbiased estimates (Baraldi & Enders, 2010; Little & Rubin, 1987). For the 
kindergarten outcome data, the state of Texas allows a variety of measures to be used for the 
kindergarten reading screen (e.g., DIBELS) so data were missing because we only collected data for 
the two most common assessments (i.e., TPRI and TejasLee). For the facility report data, it is 
possible that the pattern of missingness is related to the school’s training practices (i.e., not missing 
at random, MNAR). However, because those same schools completed the rest of the application 
and required their teachers to complete the teacher self-report, we considered it unlikely. Therefore, 
in order to retain the variability in the teacher self-report, we decided to include the observations 
with missing data in our final analyses. However, to be eligible for certification, complete data was 
required. Of the 1,185 classrooms that had at least partial data, 738 had complete data and were 
eligible for certification. 

In order to avoid convergence to local maxima, the model was run with 100 random initial 
starts, and parameters were estimated using maximum likelihood with robust standard errors due to 
some mild to moderate non-normality among the five indicators. Alternative models were compared 
using two criteria: the BIC (Schwarz, 1978), which is a measure of absolute model fit, and the Lo- 
Mendell-Rubin (LMR) adjusted likelihood ratio test, which tests the hypothesis of k vs. k-1 latent 
profiles (Lo, Mendell, & Rubin, 2001). A significant LMR test indicates that the current model, with 
k latent profiles, fits the data better than the more parsimonious model with one less latent profile 
estimated. 
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Two-class model results. The first model examined was a two-class model, analogous to a 
straight pass-fail criterion. For this model, the LMR test was used to assess the necessity of modeling 
two latent profiles versus one latent profile. A non-significant LMR test in this case would indicate 
that there was no variation in the pattern of means among the five indicators. However, the LMR 
test was highly significant, LMR = 248.1 ,p < .0001 (see Table 3), and examination of the model- 
estimated means demonstrated that the profiles of the two classes were not parallel, confirming the 
necessity of a classified structure. Standardized estimated means for the two classes are plotted in 
Figure 1. Although the profiles for the preschool variables are roughly parallel in the two classes, the 
mean differences on these variables are accompanied by substantially different average outcomes in 
kindergarten. The first class, with above-average scores on both the preschool and kindergarten 
indicators, composed 78% of the sample, whereas 22% of the sample demonstrated below-average 
scores on the preschool indicators and much lower scores on the kindergarten indicators, especially 
the literacy outcome. Although no strict criterion was set a priori for the number or percentage of 
classrooms that would pass or fail certification, the 78% of classrooms that fell into the higher¬ 
performing class was considered an acceptable pass rate. 

Table 3 


Model information andfit statistics for Latent Profile Analyses. 





Conditional probabilities 

of class membership 

Model 

BIC 

LMR 

Class 

1 

Class 

2 

Class 

3 

Class 4 

2-class 

11546 

248.1** 

75% 

25% 

— 

— 

3-class 

11523 

144.0** 

61% 

21% 

18% 

— 

4-class 

11486 

77.5* 

60% 

17% 

13% 

10% 


Note: BIC—Bayesian information criterion, 

LMR =Lo-Mendell-Rubin adjusted likelihood ratio test. 
*p< .05, **p< .001 


Three class model results. To ensure that we had adequately captured the complexity of the data, 
we also explored a three-class model. The LMR for the three class model was highly significant, 

LMR = 144.0 ,p < .0001, indicating that the three-class model fit better than the two-class model, 
and comparison of the BICs was consistent with this assessment, BIC three _ class = 11523; BIC nvo _ class = 
11546 (see Table 3). Inspection of the profiles of the three classes showed that the pattern found in 
the two-class model was almost exactly replicated by two latent profiles in the three-class model (see 
Figure 2). However, the three-class model included a new latent profile with a very different pattern. 
Specifically, whereas means on the kindergarten indicators in this third class resembled those of the 
high-performing class, means on two of the preschool quality indicators were substantially lower 
than those in the low-performing class. This new, mixed-performing group comprised 11% of the 
classrooms and was drawn primarily from the high-performing group in the two-class analysis, 
whose proportion dropped from 78% of classrooms to 67% of classrooms in the three class 
solution. The percentage of classrooms in the low-performing group dropped from 22% to 21%. 
Table 4 shows classification status and average kindergarten outcomes for the 3-class model. 
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Table 4 


Latent Profile Classification and Average Kindergarten Outcome Scores (in original units j from 3-class model. 


Class 

n 

Classroom Reading Category 
Percent "Developed" 

Classroom 

Social Screener 

High 

497 (67.3%) 

80% 

4.2 

Mixed 

84 (11.4%) 

77% 

3.9 

Low 

157 (21.3%) 

25% 

3.5 


Lour class model results. The final model explored was a four-class model. This model fit the 
data better than the three-class model, BIC = 11486, LMR = 11.5,p — .034 (see Table 3). The 
differences between this model and the three-class model centered around distinctions among the 
newly-created mixed-performance group. In other words, the percentages of classrooms assigned to 
the high- and low-performing groups did not change with the addition of a fourth class, and the 
mixed-performance group was about evenly divided into two new groups in the four class solution. 
However, since this finer-grained distinction did not add to our ability to dichotomize classrooms 
into certified or not certified groups, we opted for the three-class model in the interest of parsimony. 

Issues regarding mixed-petforming classrooms certification. Because certification by definition requires 
a pass-fail criterion or criteria, we had to decide whether or not to certify the mixed-performing 
group. On the one hand, the kindergarten performances of the children in these preschool 
classrooms resembled those of the high-performing group. In other words, the children from these 
preschool classrooms were school ready based on their kindergarten literacy scores and kindergarten 
social scores. On the other hand, these preschool classrooms did not possess the characteristics that 
are generally associated with positive kindergarten outcomes. Given the common goal of the Texas 
Education Agency and the State Center for Early Childhood Development to promote both school 
readiness and high quality early childhood education, only those classrooms empirically assigned to 
the group with high scores on all indicators were ultimately certified as “School Ready”. Thus, in this 
first year of the School Readiness Certification System, school ready certification was awarded to 497 
preschool classrooms across Texas, which represented 67% of classrooms eligible for consideration 
for certification (i.e., with complete data). 

Validity of the three class model results. In order to examine the validity of the three-class model, 
several follow-up analyses were performed. Common practice dictates that follow-up analyses aimed 
at identifying differences among latent classifications be performed in an effort to validate the 
model selected. First, ANOVA was used to compare the three groups’ means on the indicators of 
the latent profiles. Because this involved three simple contrasts for each of the five indicators, a 
Bonferroni correction was made to the alpha, p — .05 / 3, so that p = .0167, was considered 
significant. With this conservative alpha, twelve of the fifteen contrasts were still statistically 
significant, t’s = 2.99-34.83, p’s < .01, indicating that the three latent profiles indeed reliably differed 
on almost every indicator. However, the mixed performance group’s mean on the Professional 
Development in Core Subjects indicator did not differ significantly from either the low-performing 
group, t = 1.90, p = .057, or the high-performing group, / = 2.02, p = .044. The low- and high- 
performing groups did differ significantly on Professional Development in Core Subjects, t — 5.36, p 
< .0001. Additionally, as suspected, the means on the literacy outcome did not differ between the 
mixed-performing group and the high-performing group, t — 1.90 ,p — .094. We also compared the 
average attendance of the three groups. The low-performing group averaged significantly fewer days 
than both the high- or mixed-performing groups (147 vs. 156 and 154, respectively, ps < .05). 

Finally, we extended our examination of the validity of the classification scheme to include 
comparing classrooms that were ultimately certified (i.e., the high-performing group) to those 
classrooms that were ultimately not certified (i.e., the mixed- and low-performing groups). 
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Classrooms certified as “School Ready” had 55% more classmates score “developed” on the state 
mandated kindergarten reading screeners than classrooms that were not awarded certification, t — 
18.92,_p < .001. Similarly, children who attended “School Ready” classrooms had significantly higher 
scores on the measure of social adjustment, / = 9.82 ,p < .001. Finally, children in “School Ready” 
certified classrooms had higher attendance than children in non-certified classrooms, / = 3.24, p < 
. 01 . 

Follow-up Analyses Addressing the Composition of the Latent Profiles 

Because the goal of the certification system was to identify characteristics of programs that 
promoted school readiness, independent of the type of service delivery model (i.e., public school 
pre-kindergarten, Head Start, childcare), service delivery model and variables confounded with 
service delivery model were explicitly precluded from the analysis used to inform the certification 
criteria. That is, systematically making certification more or less difficult to obtain for particular 
types of programs would undermine the objective of a statewide certification system. Nonetheless, it 
was of interest to the Texas Education Agency to discover how likely classrooms from different 
service delivery models were to become certified. In addition, the Texas Education Agency was 
interested in examining the rate of certification of classrooms in the state’s integration model. 
Therefore, two sets of follow-up analyses were conducted that examined the relation of latent 
classification with facility type and the relation of latent classification with whether or not a program 
was enrolled in the Texas Early Education Model (TEEM). Finally, two analyses were conducted in 
order to examine the distribution of at-risk children into certified classrooms. 

Association between certification and service delivery system. For analyses that examined the extent to 
which latent classification was associated with type of preschool facility (i.e., public school based 
pre-kindergarten, Head Start, or childcare; see Table 5), a 3 x 3 contingency table analysis was 
conducted and found significant, (4, n — 684) = \\3.1,p < .001. 

Table 5 


Frequency (%) of Fatent Profiles by Facility Type. 


Facility type 

Latent profile 
High 

Low 

Mixed 

Childcare 

87 (52%) 

30 (18%) 

50 (30%) 

Head Start 

111 (59%) 

64 (34%) 

13 (7%) 

Public School 

273 (83%) 

40 (12%) 

16 (5%) 


Examination of cell frequencies revealed that public school pre-kindergarten classrooms 
were overrepresented in the high-performing group and were underrepresented in the low- 
performing and mixed-performing groups. Head Start classrooms were overrepresented in the low- 
performing group. Finally, childcare classrooms were overrepresented in the mixed-performing 
group and were underrepresented in the high-performing group. 

Because service delivery system only varied among classrooms that were enrolled in the state 
integration program, a restricted analysis was performed in which we examined the relation of 
facility type and latent classification only among classrooms enrolled in the state integration 
program. The results of this restricted analysis were largely the same as those performed with the 
entire sample of classrooms. The only exception was that when childcare classrooms that served 
higher income children (i.e. classrooms that were not enrolled in the state integration program) were 
removed from the analysis, childcare classrooms were no longer underrepresented in the high- 
performance group. 
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Association between certification and participation in state integration model. Two sets of follow-up 
analyses were conducted to examine the extent to which latent classification was associated with 
enrollment in the state integration program. First, a 3 (Latent profile: High, Low, Mixed) x 2 (State 
program model: Participating or Non-participating) contingency table analysis was conducted on the 
full sample. However, in light of the fact that all of the classrooms in our sample that did not 
participate in the state integration model were classrooms in childcare centers, we conducted a 
second, restricted analysis on a sample that only included childcare settings. 

For the full analysis, there was a significant association between classification into the three 
types of latent classrooms and enrollment status in the state integration program, yf (2, n — 731) = 
80.2 ,p < .001 (see Table 6). Specifically, TEEM classrooms were slightly underrepresented in the 
mixed-performing group. The opposite was true of classrooms that did not participate in TEEM, 
which were overrepresented in the mixed group and were underrepresented in both the high- and 
low-performing groups. 

Table 6 


Frequency (%) of Patent Profiles by Program Status. 



Latent profile 



Program Status 

High 

Low 

Mixed 

Not State Program 

21 (42%) 

4 (8%) 

25 (50%) 

State Program 

472 (69%) 

151 (22%) 

58 (9%) 


For the restricted analyses, the only significant deviation from expected cell frequencies was 
an overrepresentation of classrooms that were not enrolled in TEEM in the mixed-performing 
group, yf (2, n — 167) = 13.8,/) < .01. These results indicate that for-profit child care programs that 
served middle income families were more likely than childcare programs enrolled in TEEM to 
evidence poor quality preschool programming but still graduate children who were ready for 
kindergarten in terms of academic and social preparedness. 

Association between certification and at-risk populations. We also examined the association between 
classroom certification and children’s free/reduced lunch status, an indicator of economic 
disadvantage. No differences were found in the distribution of disadvantaged children into certified 
classrooms, yf (1; n ~ 10,137) = .86, ns (see Table 7), indicating that economically disadvantaged 
children were equally likely to be in a certified classroom as their more affluent peers. The second 
analysis compared certification status between native English speakers and English-language 
learners. Results indicate that English-language learners were more likely than expected to be in a 
not-certified classroom, whereas native English speakers were more likely than expected to be in a 
certified classroom, yf (1, n — 5,376) = 75.0 ,p < .001. 

Table 7 


Percentage of free / reduced lunch and English language learners by certification status. 


Class 

n 

Free or reduced lunch 

English language learner 



percent 

percent 

Certified 

7,435 (73.4%) 

74.3% 

44.3% 

Not certified 

2,702 (26.6%) 

75.2% 

59.4% 
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Discussion 

This project was initiated by a state mandate to develop a system that would inform parents 
about which early childhood programs were preparing preschool-age children for school. Key 
stakeholders agreed that the resultant quality rating system should take into account both 
scientifically-based indicators of quality pre-kindergarten programs and indicators of children’s 
school readiness. To our knowledge, this is a unique approach that has not been implemented by 
any other state or national quality rating systems, as most quality rating systems only consider 
characteristics of the pre-kindergarten program and classroom environment without linking these to 
kindergarten outcomes. A goal of the system was to ensure applicability of the criteria to the full 
range of early childhood service delivery types and thus, characteristics that were unique to a specific 
type of program were not included in the certification criteria. 

Advantages and disadvantages of the certification system 

Working within the fiscal constraints of Texas, this approach was only possible through 
capitalizing on the situation that the public schools already performed kindergarten screenings to 
identify children potentially at risk for reading problems. Constraints on development of the 
certification system included: (a) a need for cost-effectiveness, (b) reliability and validity of self- 
report data, (c) inclusion of indicators of quality of pre-kindergarten program and indicators of 
children’s school readiness, and (d) development of an unbiased system that could be used by all 
early childhood education service delivery systems. Thus, the School Readiness Certification System 
(SRCS) strives for a balance between good science and the financial and political realities faced by a 
new state-wide program. 

Advantages of the SRCS are that it is a web-based application that allows input from the 
teachers, program directors, and/or principal. While input on a web-based system is cost-effective, a 
challenge is assuring that the information collected is valid in that it reflects actual classroom 
practices and the classroom environment. In a pilot study, reliability and validity of teacher-self¬ 
report data was corroborated through observations of classrooms by external evaluators. Whereas 
annual on-site visits to all programs is cost prohibitive in a large state with thousands of early 
childhood education programs, randomly selected observations by external evaluators to verify 
validity of data in applications is feasible for future years. Additional support for the validity of the 
data entered into the web-based application comes from the fact that these data were related to 
children’s school readiness in ways expected from prior research. 

Determination of the preschoolfactors. Decisions about items to include in the web-based 
application were based on indicators of quality obtained from research studies that include 
classroom environment, instructional practices in social and cognitive skill domains, curriculum, 
assessment approaches used to monitor children’s progress and professional development (e.g., 
National Research Council, 2001; Whitehurst et al., 1994). With this broad range of information in 
combination with children’s outcomes in early kindergarten, it was possible to determine the most 
important pre-kindergarten factors to emphasize in a quality rating system. The results show that it 
was possible to refine the information such that a set of unique indicators from the pre-kindergarten 
period could be identified as most important in terms of relations with reading and social 
competence in early kindergarten. However, it is recognized that the development of this system 
does not allow for causal inferences as there were too many unobserved intervening variables that 
cannot be ruled out. 

Results of the latent profile analyses showed that three constructs in the pre-kindergarten 
environment provided the best information for discriminating classrooms into distinct groups. 

These indicators were: (a) teacher professional development, particularly core training in school- 
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readiness (e.g., rich language input, early literacy, social competence); (b) teacher intentional 
instructional approaches that promote achievement; and (c) literacy oriented instructional activities 
especially early writing activities. Together these aspects of the pre-kindergarten environment 
discriminated, or predicted, those classrooms whose children had higher vs. lower early kindergarten 
social and reading outcomes. What is particularly interesting is that kindergarten social and reading 
scores and the three pre-kindergarten quality indicators were positively correlated. These findings 
have implications for efforts to understand how intentional cognitive activities during the pre¬ 
kindergarten years may support a range of school readiness skills including social competence 
(Duncan, Dowsett, Claessens, Magnuson, Huston, Klebanov, et al., 2007; Reynolds, Mavrogenes, 
Bezruczko, & Hagemann, 1996). They suggest that classrooms that provide planful, cognitive 
activities such as early literacy carried out in intentional ways may also promote children’s social 
competence. While the indicators have a focus on building learning in language and literacy areas, 
the professional development and intentional instruction areas included responsive interaction styles 
between teacher and children as well as support for children’s efforts. This included rich language 
input such as the use of rare vocabulary words, providing explanations and expanding children’s 
utterances. Taken together, these indicators suggest that classroom teachers can incorporate 
practices that promote both social and cognitive competence and that the two areas of development 
do not involve a tradeoff. 

Validity of the certification system. Another key finding was that children from poverty 
backgrounds who were in well implemented integration classrooms got similar reading and social 
pass rates to children from higher income backgrounds in terms of their early kindergarten screening 
scores. Moreover, they demonstrated better social competence, on average, including the ability to 
regulate their behavior in social situations. One explanation for improved social regulation for these 
children could be the emphasis on TEEM teachers providing predictable routines and engaging 
learning activities. This may provide children with the support they need to learn to more 
independently organize their behavior in activities that require cooperating with others and taking 
initiative. The formula for certification provide support that children living in poverty, whether 
enrolled in child care, Head Start or public school classrooms can enter kindergarten with the 
reading and social skills necessary to learn what is required in kindergarten. This finding is consistent 
with a conclusion reached regarding twins raised in different environments, where 60% of the 
variance in cognitive skill was related to environment with genetic influence being essentially zero 
(Turkheimer, Haley, Waldron, D’Onofrio, & Gottesman, 2003). These results counter the arguments 
that children from poverty backgrounds cannot reach similar levels of achievement with that of 
more affluent peers when given the right environment for learning. It is encouraging that exposure 
to high quality classroom-based program that emphasize cognitive readiness (e.g., language, literacy 
building) in ways that supported social competence may be able to place children from low SES 
backgrounds on an equal footing with more economically advantaged children. However, it is also 
possible that the classrooms in the mixed class that mostly served middle-income children were not 
supporting these higher SES children’s learning adequately. 

It is important to note that classrooms from each of the service delivery programs were 
classified as high quality, albeit classrooms from public school programs showed the highest rate of 
certification. This might be expected as public school requires a college degree though the results 
show that this alone did not assure 100% certification status. The results of the certification of the 
Head Start classrooms was somewhat lower than that of public school classrooms, and this may be 
due, in part, to the diversity of performance standards, or goals to which such programs attend as 
well as a lower income criteria that often means children in Head Start programs are from 
particularly disadvantaged backgrounds. This said, subsidized childcare and Head Start classrooms 
showed greater than 50% certification rates in spite of serving very low income children. 
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Secondary benefits ofithe certification system. While the goal of the School Readiness Certification 
System was to identify classrooms that prepare children for entering kindergarten, it also has the 
potential to serve as a guided self-assessment. Technical assistance for programs that did not meet 
the certification criteria was available and considered a key component of the system. Thus, it serves 
a supportive rather than a punitive role. For those not certified in the present study, a profile of 
strengths and weaknesses was developed from the data base and after each program received this 
report, they were offered the opportunity to discuss a technical support plan with the State Center 
for Education and Child Development, who developed the SRCS. Specifically, state-funded early 
childhood support staff in each community partnership are trained to interpret this profile and to 
prioritize a sequence of steps for supporting the classroom to improve weak areas. The advantage of 
the technical assistance is that the support is tailored to the needs of individual programs so that 
resources have a high likelihood of being used effectively. This better assures that when a program 
reapplies, it is more likely to achieve certification. 

Given the voluntary nature of the certification system, a question arises as to what the 
incentive is for programs to participate? For school districts, a strong advantage of a comprehensive 
quality rating system is that it has the potential through market forces to encourage more children to 
arrive at kindergarten ready to succeed in public school. This would allow for better use of resources 
in kindergarten and beyond, since fewer children would require referral to special education or need 
remedial services. Also, less of the initial part of the kindergarten year would be spent by regular 
education teachers getting lower achieving children up to speed with the rest of the class. For 
stakeholders, such as directors of large child care agencies or Head Start agencies, feedback provided 
through a comprehensive quality rating system would allow for an increased awareness of which 
individual programs were more or less effective. This information can serve functions of 
accountability as well as guide program development. For individual centers and programs, receipt 
of a high rating or certification at minimal investment cost would allow positive advertisement 
within a community of program effectiveness. For example, private for-profit childcare programs 
need to remain competitive for continued funding. One means to demonstrate a competitive edge 
would be to market oneself as a program that has been independently determined to promote school 
readiness. 

Limitations and future directions 

Practical considerations that need to be addressed through ongoing research with this system 
include the issue of how to classify programs with multiple classrooms where some reach 
certification criteria and others do not. Ideally, all classrooms at a given school would be assessed 
and certification would be awarded to schools rather than individual classrooms. Within the current 
context of classroom-level certification, a possible solution would be to certify a program as Texas 
School Ready if a majority of classrooms reach certified status. This could be combined with the 
criteria that the minority of non-certified classrooms within a program reach certification within a 
specified period of time. An important policy implication includes the need for states to find 
sustainable funding for such a system so that no program is excluded due to lack of financial 
resources. 

There are some important limitations to the current study that are noteworthy. Data from a 
group of classrooms could not be included in analyses that informed the certification criteria 
because these classrooms had incomplete applications. Additionally, it is important to note that if a 
group of children scored developed in reading and social skills in kindergarten, but the preschool 
classroom that they attended did not have scientifically-based best practices in place, then those 
classrooms were not certified. This practice allowed the system to stay true to its purpose; 
identifying programs that have teaching practices in place that scientific research has demonstrated 
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promotes learning. To only base certification on kindergarten outcomes could introduce a SES bias 
into the certification process. That is, a large body of research shows that low-income children often 
score below middle income children on measures of academic achievement (e.g., Zill & West, 2001). 
This occurs, in part, because higher family income is strongly related to higher levels of parent 
education, a predictor of children’s academic achievement. Also, had we certified solely based on 
kindergarten outcomes then we would have certified classrooms in the mixed-group that did not 
provide children with the learning experiences that would have allowed their children to reach their 
optimal potential. That is, as many children in the mixed group were from middle class backgrounds, 
they might be expected to achieve even higher in kindergarten if they were provided with an optimal 
learning environment. In sum, we believe that the best approach to rating early childhood education 
classrooms or programs is one that takes into account both quality of preschool programming and 
children’s school readiness outcomes. 

An important aspect of this system is that the items in the Teacher Self-Report represent 
constructs that teachers can report in a reliable manner. Contrary to expectations, two areas that did 
not show good reliability between observation and teacher self-report was responsiveness to 
children’s needs and behavioral management. While these are considered quite important in 
descriptions of best practices, the ability to reliably capture this information could not be 
demonstrated with this system. Given that major goals of the system were that criteria had to be 
captured reliably and empirical methods needed to be used rather than beliefs regarding best 
practices, it was necessary to drop these items. Future checks on the sensitivity of the system will 
allow for consideration of additional items and measurement methods. 

Given the emphasis on self-report from program administrators and teachers, the desire of a 
school to be certified could result in a “high stakes” climate that would promote responses based on 
presenting the classroom in the best possible light. There also was concern that incorporating end- 
of-the-year testing in pre-kindergarten would be experienced by teachers as a “high stakes” type of 
testing, which may lead teachers to be overly concerned with the end-of-the-year assessment and 
they may “teach to the test” rather than attend to the whole child. This contributed to the decision 
to use the state kindergarten screening mechanism already in place rather than testing at the end of 
the pre-kindergarten year. The same concern may be applicable to specific self-reported teaching 
behaviors where, to ensure reliable reporting, the system may need to identify additional item sets 
that can be used in a random rotation approach to avoid response bias. Finally, the state’s ability to 
promote participation of pre-kindergarten programs in the SRCS will be dependent upon an 
emphasis of the usefulness of the system for program evaluation and improvement and avoidance of 
using the system in a punitive manner. 

Finally, at present the SRCS system lacks a more comprehensive set of school readiness 
outcomes. The combination of a limited set of school readiness outcomes and a data reduction 
process that excluded pre-kindergarten classroom characteristics that were unrelated to kindergarten 
outcomes may have oversimplified our characterizations of pre-kindergarten classrooms. For 
example, the Math Instruction factor was not used to help classify classrooms because it was 
unrelated to kindergarten outcomes, which was not surprising given the absence of a mathematics 
outcome. As early mathematics skills are receiving a great deal of attention from researchers and 
educators, the inclusion of a math screener as a kindergarten outcome in the system will be an 
important addition in the future. On the other hand, an asset to the SRCS is the inclusion of both an 
academic outcome (i.e., reading readiness) and a measure of social competence in its 
conceptualization of school readiness. Staying true to this whole-child conceptualization of school 
readiness was no small feat, as the social screener was not a state mandated kindergarten screener. 
Instead, collecting these data required enormous efforts and cooperation among school districts, 
principles, and teachers. While many districts were willing to have their kindergarten teachers 
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complete the social competence measure, a more seamless approach would be to legislate a common 
social screener across the state. 

In summary, school readiness for young children is promoted through a classroom focus on 
literacy, language, and social skills through the implementation of a core curriculum, professional 
development for teachers specific to the area of school/cognitive readiness; and intentional high 
quality, school readiness instruction. The development of the SRCS demonstrates how an 
empirically based approach can result in an effective means for informing parents, state policy 
makers, and educators about the quality of early childhood programs. It will be important to 
remember that such a system is a dynamic process that will require ongoing attention and revisions 
that are responsive to new research findings, the continued sensitivity of the quality indicators, and 
expansion of early kindergarten outcomes (e.g., math). To assure knowledge about whether 
programs can sustain quality over time, the system will require re-certification on an annual or bi¬ 
annual basis. 
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Appendix 

This section provides a description and photograph scoring examples for two classrooms. While 12 
photographs per classroom were considered when scoring the Preschool Environment Checklist, 
only 6 photographs that best represent key distinctions in scoring criteria were selected for the 
purposes of this illustration. Classroom 1 embodies many of the characteristics consistent with high 
scores, whereas Classroom 2 represents an environment receiving low scores. 


Classroom 1 

Central to the scoring system is the presence of several key visual supports, including a daily 
schedule, jobs chart, daily news, center management systems, rules chart, letter walls, and read aloud 
charts. With the exception of the read aloud chart, each of these elements is visible in Classroom 1. 
Once identified, coders consider the quality of these supports (e.g., at eye level, interactive, linked 
with learning objectives), which in this case fall into the high range. This classroom also receives 
high scores on a series of items connected with the richness of the learning environment, including 
access to a variety of books and writing materials, as well as a centers-based room arrangement that 
supports peer interaction and exploration in a variety of content areas. See Classroom 1 photographs 
below for specific examples. 
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Classroom 2 

This classroom is characterized by a lack of stimulating materials, print and interactive 
management systems. For example, several key visual supports captured with the measure are 
absent, including a daily schedule, rules chart, jobs chart, letter wall, and daily news. While evidence 
of a center management system can be seen (e.g., construction center label with space for children’s 
names), the management chart appears inaccessible in the dramatic play center. In such cases the 
center management ratings would fall in the mid-range, allowing high scores to be reserved for those 
management systems that apply to several learning areas and appear to be used routinely. Classroom 
2 also receives low scores on the series of items reflecting the richness of the learning environment 
as evidenced by a lack of books, print, and variety in centers-based learning opportunities. See 
Classroom 2 photographs below for specific examples. 
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